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Engineering — Building Technology — Fire Research — Chemical Engineering? 
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economy in Government operations in accordance with Public Law 89-306 (40 U.S.C. 759), 
relevant Executive Orders, and other directives; «carries out this mission by managing the 
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From a survey of current tool usage It is eabetiaed that’ the greatest . 
obstacles tp effective use of software tools are encountered In - 
organizations employing fewer than 40 programmers , and the needs of 
these environments are therefore emphasized.. Specific:'needs for’ 
software tools In programming for management Information: systems ‘and 
for sclentific applications are discussed. Measures are described to: 
overcome organizational obstacles to use of tools, to. deal with 
problems arising from the tools, and to reduce the, difficulties posed 
by ex| sting computer parley 


In two ways:-by the function responsible for thelr accomp|ishment, . and 
by the time schedule In which they must be completed. The detal | wor 
to be performed | In each step Is: ase (hea, & : 


- 


Steps required for the successful Introduction of tools ‘are set 


Key words: . conpiikay a ee software; software “shgtneehinas: 
software management; software quality; software tools; toolsmith. 
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SECTION 1 : 
EXECUTIVE SUMMARY 


oS 
This publication 1s Intended to. provide guidance for the Introduction of 
software tools for agenclés of the U. S. Government and for computer users at - 
-large. It Is primarily almed at Installations where there had been IIttle or no 
use of software tools previously. In a survey of software tool usage It was 
found that the slze of..the programming group had a significant effect on the 
extent of tool usage, with organizations of less than 40 programmers much less: 
likely to be tools users, To provide help to these smaller organizations In the 
hntroduction and use of software tools Is therefore one of the goals of this 
document. é 


Difficulties In the Introduction of tools can arise In three areas: 


Organi zatlonal obstacles. > 
Problems arising from the tools. 
Obstacles In the computer environment. 


obstacles can be reduced If a: responsible management level Is 
Involved In the Introduction of tools. Those who commit the resources for tool 
acquisition and use should participate actively In the relevant decisions’ 
Their Involvement In the following Is particularly Important: 


1. Identifying the goals-to be met by the tool (or by. the technique supported 
by the tool), and assigning rasponslbih ity for the activities required to 
meet these goals. 


Ls Approving a detatied tool acquisition plan that defines the resource 
requirements for procurement and In-house activities. 


3. Approving procurement of tools and tralning If this Is not explicit In the 
approval of the acquisition plan. 


4. Determining after some period of tool use whether the goals have been met. 


from the tools can be avoided by a careful, methodical 
selection af tools. In‘particular, distinct contributions to the tool selection 
-are specified for software management and the software engineer. Software 
management Is assigned responsibility for: 


NI 


Identifying tool o jJectives. 


2% Approving the acquis on plan (higher approvals may also be required). 
ae 


3. . Defining selection criterla. 


4. Making the final selection of the tool or the source.. 


- 


The software eng! neer Is responsible for: 
1. Identifying candidate tools. 


2. Applying the selection criteria or preparing technical: sections for 
@ Request for Proposals (RFP). 


3. Preparing a ranked IIst of tools or sources. : 


Further, the ultimate user of the tool should be Involved in reviewing el ther 
the list of candidate tools or, for formal procurement, the tool requirements. 


Obstacles In the computer environment are primarily due to the great diversity 
of computer architectures and, operating system procedures, and to the lack of 
rtabllity of most software tools. Activities assoclated with the Introduction 
i tools can only modestly alleviate these difficulties. Guldance Is provided 
or: 


‘ » 

1. A methodical process of Identifying candidate tools and selecting among 
these on the basis of established criteria, Including a definition of the 
computer Interface. This wlll avold some of the worst pitfalls assoclated 
with "borrowing" a tool from an pe area or procuring one from the most 
accessIble tool vendor. 


\ é 

2. The assignment and training of a toolsmith who can make minor modifications 
to both the computer environment and the tool. This Is expected to provide 
rellef where there are verslion-related or release-related Incompatiblil ities 
with the operating system, or where the memory requirements of the tool 
exceed the capabllities of the Installation. In the latter case, remedies 
may be provided by removing tool en rens or by structuring the tool program 
Into overlays. 


As part of this work, an event sequence for the tntroduction of tools has been 
developed that Ident fles specific tasks, the assignment of responsibilities for 
the tasks, and the order In which they have to be carried out. 


SECTION 2 8 
INTRODUCT 1 ON en ie. 


This publication Is Intended to provide guidance for the Introduction of 
software tools for agencles of the U. S. Government and for computer users at 
large. It Is primarily almed at Installations where there had been IIttle or no 
use of software tools previously. In a- survey of software tool usage It was 
found that the sl#e of the programming: group had a significant effect on the 
extent of tool usage, with organizations of less than 40 programmers much less 
likely to be tools users [HECH81]...In particular, organizations of less than 40 
programmers were found jto need help In, order to acquire and employ software 
software tools successfully, and the requirements of these organizations are 
glven special emphasls. 


Many of the difficultles reported by novice users with software tools can be 
overcome by systematic practices In the selection, acquisitvon, and preparation 
for use of software tools. This report first derlves the need for specific 
guidance In the Introduction of tools by examining a number of programming 
environments, and then describes the practices sulted to these environments. 


Sectlon 3 charaterizes user environments In terms significant for the 
Introduction of software tools. In thls characterization, two environments were 
Identified that will benefit most from formal guidance for the Introduction of 
tools, and a vignette of each ofthese Is presented In the final parts of 
Section 3% \ 
, q 

Tool needs for varlous user environments are described In Section 4. First, a 
falrly broad discussion of organizational and application factors that govern 
tool needs Is presented. Then, based on these considerations, a generic 
(features based) Identification of tool needs for the two target environments Is 
made, Needs of other environments are also discussed, and special attention Is 
focused‘on the Integration of tools. The final. part of Section 4 covers 
resources for the selection of tools. The recent publication of a report on 
software development tools by NBS/ICST Is of major assistance In this area 
[HOUG82]. The generic software tool nomenclature used In the present report Is 
taken from [HOUG81] which In turn Incorporates major portions of a Software Tool 
Taxonomy [REIF80]. 


The time phasing aspect of the Introduction of toals Is described In S lon 5 
by means of event sequences. The purpose of event sequences Is (@lseussed In 
general terms, and the specific event sequence for the Leek soffware 
tools Into the smaller programming environments Is then developed. The events 
are cléssifled by area of responsibility and precedence relationsh!ps. and each 
of the required events. Is described In detall. 


A preliminary draft of thls document was discussed at a Workshop on PhasIng of 


Software Tools which was held at NBS on 18 May 1981, The agenda and the: 
attendance |Ist are reproduced In the Appendix, The participants contributed 


g 
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many constructive comments which have been Incorporated Into the present 
version. Written comments were recelved from several Individuals who could not 
attend the workshop; these contributors have been |Isted as "reviewers" In the 
AppendIx, ‘ 


The author wishes to acknowledge the contributions of collaborators In the 
preparation of this document. Myron. Hecht analyzed the survey results which 
form the basis for Section ,\3, and Donaid J. Relfer classified tool needs as 
reported in Section 4. Much helpful guidance In the conduct of this study was 
recelved from the technical monitor for the contract, Mr. R. C. Houghton, Jr.* 
Continued encouragement and many helpful suggestions were furnished, by Dr. 
Martha Branstad, the ICST Software Quality Program Manager. 
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SECTION 3 
CHARACTERIZATION OF USER ENVIRONMENTS 


This section considers the characterization of user environments along IInes 
that are significant for the Introduction of software tools. The starting point 
for this characterization Is the classification of software tool users which Is 
summar|zed In subsection 3.1. ‘The selection of target environments for the 


Introduction of software tools, based on this classIfication, Is deseribed In: ° 


subsection 3.2. . The smaller classes of management Information system, (MIS) and 
sclentific programming environments are Identified as most In need of outside 
assIstance In tool usage, and vignettes typical of each of these environments 
are presented In subsections 3.3 and 3.4, respect Ively. 


3.1 CLASSIFICATION OF ENVIRONMENTS 


A Survey of Software Tools Usage [HECH81] considers the effect on tool usage of 
a ia large number of shiv ponesnval factors. Including: 


Size of software: seeileation: 

Type of organization (private, Government-support, Government). 
Applications (scientific, MIS) and language. 

Development environment (batch, Interactive). 

Program running environment (batch, Interactive, real-tIme). 
Computer type. / 

_ Involvement In tool development. 


3 


"The first and last factors were found to have a significant effect on the extent 


Of tool usage. The type of. organization was not found to be a major determ! nant 
of the’extent of tool usage In this survey. The other factors had some effect 


" [on the types of. tools that were used but not on the extent of tool usage (or the 


effect was masked by correlation with primary determinants of tool usage). 


In the followlng discussion the extent of tool usage Is classified Into three 
lévels: Be Oe, : 


‘Level 0 - Minimal tool usage - only tools normally provided with the operating - 


system were In-use Sgecenb lors: loaders, compilers, debug alds, and 
Interpreters). ~! 


Level 1. Intermediate tool usage - special purpose. todis sulted for the mission 
of the orgahization but without explicit effect on software quality 
were In use, Examples are simulators. file managers. and elementary 
precomplilers.: 


- general purpose tools, Involving static 
and dynamic analysis features. were del'lberately acquired or developed 
In order to enhance software quality and productivity. This group 
represents the highest level of tool utllization identified In the. 
survey. ; 


By Interpreting the level . Index 7 1, or 2) as a number, an average level of 
tool utIlIlzatlon can be computed for groups of-tool users. The average level of 
tool utilization as affected by the size of the organization Is shown In Table 
3-1. 
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TABLE 3 - 1 LEVEL OF TOOL UTILIZATION 


Size of Organization Avg. Level of 
Too] UtH Ization 

Small - up-to 14 sragrenners i os is 

Medium - 15 to 39 programmers 0.8 

Large ~ 40 to 99 programmers 1.4 

Very large - over 100 progranmers 2.0 


The term programmer Includes analysts, programming supervisors. and programming 
tralnees but not computer operators, Iibrarians, or other support personnel. 
The above data are based on a survey of 22 organizations. Tool developers were 
not Included In this POPUIET ttn 


3.2 SELECTION OF TARGET ENVIRONMENTS - 


. As can be seen from Table 3-1, the use of general purpose software tools was 
considerably less prevalent among smal! and medium software organizations than 
among the large and very large organizations. In all size classificatjons there . 
was representation of private, Government, and Government-suppdrt organizations 
(the three classIflI-catlons for type. of organization considered In this study). 
No evidence was found that the organizatlom’type affects, the level of ‘tool 
usage, but because of the smal | ae size this. Is regarded as only a tentative 
conclusion. } 4 


These data IndIcate that smal! and medium software organizations will represent 
the target environment that stands to benefit most from the avallability of a 
comprehens lye methodology for ‘the, Introduction of software tools. In addition 
to the low level of current tool usage shown In Table 3-1, ,the following factors. 
Indicate that small and medium organizations need outside agst stance In the 
Introduction of tools: 


1. ~thetr awareness of tools In general, and thelr knowledge about specific 
tools sulted to thelr needs, are frequently much less than That of larger 
Organizat tons. 
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Thelr knowledge of tool acquisition and Iristal lation practices tends. to be 
Inadequate to permit them to obtain the full benefit from avallable tools. 


Even when suitable tools are obtained and Installed, these organizations - 

frequently cannot mobilize the resources required for optimum tool 

utilization, such as training, al efforts, and ia ale In practices to 
_ fully utilize a tool. ; 


A further consideration (which ‘partly encompasses all of the above) Is that a 
given level for effort In developing a methodology for the Introductlon of tools 
.can be expected to provide much more significant and measurab | results, If that 
effort Is targeted at organ zations at the smaller end of the size range. 


The above does not Imp ly: that large, and even very large, organtzations cannot 
benefit from further developments of methodology for the Introduction of 
software tools, and specifically from efforts In that area undertaken by 
NBS/ICST. The needs of these environments are further addressed In subsection 
4.3 of this report. 


The need for outside assistance for the development of a sultable Introduction 
abe Is shared by small and medium size organizations. There are only 
minor différences In the detalls of the application of the methodology between 
small and medium size organizations, and to avold long titles the term "smaller" 
will henceforth designate the two groups collectively. Within the smaller size 
_ groups, the Introduction methodology will focus on Government organizations 
~ although. as will be explained shortly, most of the Introductory practices are 
not expected to vary significantly as a function of the organization type. The. 
reasons for focusing on Government organizations are: 


1. The demand for uniformity of software practices In Government agencies Is 
expected to Increase, and tools can be of assistance In providing and 
ctng this untformity. Hence, a greater need for tools Is expected to 

arlse \n thls environment. ee RA 


Government agencies usually have a greater ‘need to control procedural 

aspects software development, and many tools address that need very 

specifica ; 

t 

There are a large number of tools currently In Government ‘Inventory, and 

some, of these are resident on computers that can be accessed by other 

Government organizations vila terminals. Experience with tools may be 
’ shared, and help with tool problems may be furnished more readily among 

Government agenctes than within the private sector or between Government 

and private organizations. Thus, the opportunity for tool usage Is greater 
among Government organizations. 


Successful use of a tool In a Government organization Is IIkely > become 
generally known (via professional organ}zations, computer user groups, 
etc.) whereas smaller private organizations may wish to restrict the 
dissemination of this Information for competitive reasons. Thus, the 
‘ ripple effect can be expected to be greater If Government organizations are 
addressed as the primary target for the tool Introduction methodology. 
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Except for the factors mentioned above, the activities and level of effort 


' required for the Introduction of tools are not believed to be significantly 


different among private. Government=-support, and Government organizations, The 
greater avallabllity of tools may appear to confer a material advantage on 
Government organizations but at present this has not been a cause for Increased 
usage. The annual licensing fee for a typical tool. Is of the order of $ 1,000, 
and purchasing costs are five to ten times that amount. These are usual ly not 
_the dominant expenses In the Introduction of a software tool. A large number of 
tools are In the public domain and coples can be obtalned at nominal cost, from 
computer vendors, universities, and some organizations which specialize in this 
fleld. Thus, although the terminology used In the following may be specific to 
Government organizations, the general concepts are belleved to be broadly 
applicable, : 


Among the smaller Government organizations, the survey found differences In 
tool needs that Indicate that administrative and scientific environments may 
best be treated separately for some aspects.of the Infroduction of software 
tools. A demonstrable difference Is In the types of tools needed (In turn 
dictated by the languages used); the most widely encountered tool In smaller 
sclentific organizations Is a FORTRAN preprocessor. whereas COBOL egyironments 
frequently .use optimization tools that have no direct counterparts In the 
sclentific environments, A more’subtle difference exists In the overall 
attitude towards tools. Sclentific programmers (specifically engineers and 
sclentists doubling as programmers) know about tools and’ may be consclous of 
some of the advantages that they confer, but are Interested primarily (sometimes 
exclusively) In solving sclentific or engl neeering problems, They are only 
slightly motivated to devote any effort toward the enhancement of software 
quality. Programmers and. first level supervision In the smaller administrative 
or MIS (Management’ Information Systems) environment may be only vaguely aware of 
tools but are highly motivated to Improve the quality of thelr software, 
‘particularly Its. Beisel neers . 


— The following subsections provide vignettes of lig smaller MIS and sclentific 


environments, respectively, that peer emphasize factors pertinent In the- 
Introduction of tools. 


~3.3 THE SMALLER MIS ENVIRONMENT 


The term MIS environment Is Intended to Include all gramming for fiscal, 
administrative, housekeeping, and record-keeping functlons. The predominant 
language Is COBOL but a falr amount of assembly language programming (In 
application programs) Is also In use. ALGOL and PL/1 are used occasionally Ina . 
few agencies. Practically all system programs used by the smaller MIS ~ 
organizations are written In assembly language. 


- By our definition, a smaller organization: may Include up to 39 programmers, but 
the representative Government organization In this category rarely..Involves more 
than 25 or 30 programmers. It Is typically a fleld office or a central 
programming organization for a special Ized agency or function within an agency. 
There are two levels of ws se The lower one deals with a specific 
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programming area (systems, dIsbursals, security, etc.) while the major 
responsibility of the upper supervisor Is to malntalin |lalson with the 
headquarters organization which generates the requirements and funding for the 
office. Yery few, If any,-of the smaller Government MIS organizations can make 
It a major assignment for one of thelr employees to provide guldance In ‘software 
technology and programming practices. Some of this guldance might be provided 
by, headquarters organizations, and thus will be relayed through the highest 
- supervisory level. But without a specific local designee who provides 
follow-up, much of the Impact of headquarter guidance will be lost. 


The range of programmer skiIl levels encountered In MIS organizations Is broader 
than that prevalent In sclentific envlronments, primarily due to the use of 
programmer tralnees by the MIS organizations. The formal tralning of the 
programming tralnees consists of In-house courses, technical school courses, and 
approximately 1 year of attendance at a community college. They are tralned for 
program writing rather than software design or broader aspects of computer 
sclence or software engineering. There Is also little Involvement In standards 
or professional activities among the MIS organizations. Indicating few 
opportunities for a continuing, broadening education of the programmers In this 
environment. ‘ 

The primary activity of the smaller MIS organizations frequently Is program 
maintenance, The programs undergo almost constant change due to: 


Changes In legislation. 

Changes In administrative procedures. 
Major organizational restructuring. 
Program or functional Improvement. 
Correction of errors. 


Offices have backlogs that range up to 1 year. Malntenance Is a slow and 
‘ difficult process because of the lack of good documentation (a factor that 
transcends this environment), the low skIl! levels, and.the lack of good tools. 
When aval lable, tools may be used very effectively, e. g., the use of a file 
manager for configuration management of the programs, or full employment of the 
features of a sophisticated editor. 


In general, the smaller Government MIS organizations do not tack motivation for 
tool use-and make use of avallable tools. Frequently, however, they lack both 
the knowledge and the resources to use tools more effectively. They will 
benefit from outside assistance In all of the areas Identified In 3.2 above. 


As far as tool needs are concerned, the MIS environment presents some unique 
problems, e.'g., the lack of portability of most’ COBOL programs. and the 
run-time Inefficiencles caused by most commercial COBOL compilers. The first 
problem prevents agencies from sharing application programs. even where the 
purpose served and the records to be generated are Identical, unless they also 
have the same computer. The lack of portabllity Is more of an obvious problem 
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In this environment than In the. sclentIific one because many applications are 
common to practically every business environment: payroll, budgeting physical 
asset management, and billing. Among Government agencles there are further 
commonalities due to Government regulations, Interaction with the General 
Services Administration, .and réquirements of the Office of Management and 
- Budget. In addition to the obstacles which the lack of portability represents 
to the Interchange of programs, It also creates great problems If a computer Is 
belng replaced by a more capable model from a different manufacturer. 
Conversion activity due to replacement of a central computer complex can extend 
over several years and represent resource expenditure comparable to the hardware 
cost. 


The run time inefficlencles of COBOL compllers could be tolerated In the past 
(at least In some cases) when computer programs were used to generate perlodic 
hard copy reports which would then serve as the user's primary data base. The 
predominant practice today Is to update these reports continuously and to make ~ 
them avallable to the user ‘6n Interactive terminals rather than In printed 
format. This puts a much higher premium on efficient execution of programs 
because of the more frequent access and the need for a rapid response to user 
requests, To meet the demands of the current environment, elther more computers 
_have to be Installed or the efficiency of the object code has to be Improved. . 
The latter approach has some obvious IIimitations, but within these It Is a much 
more cost-effective way of Improving the performance of a computer complex. 
Optimization programs of several types are avallable to deal with this problem. 
These are not commonly in use In smaller organizations of any type but they are 
frequently encountered In larger MIS organizations. 


3.4 THE SMALLER SCIENTIFIC ENVIRONMENT 


The typical size of the smaller sclentific software development group Is also 25 
to 30 programmers, and two levels of supervision are Involved. The lower 
supervisory level tends to be application orlented but the top supervisory 
‘ function operates much more autonomously than In equivalent MIS agencies. 
Though constrained by budgets that are determined at,a’higher managerial level, 
the second level sclentific supervisor typically assumes full responsibility for 
the technology employed within his or her organization. Where this supervisor 
takes an Interest In software technology, structured programming supported by 
appropriate tools Is IiKely to be used. On the other hand, If the Interest of 
the supervisor Is confined to a sclentific specialty (simulations, engineering 
analysis), software technology can be a very low priority Item. 


Most programmers In the smaller sclentific environment have an engineering or 
sclence degree but thelr formal training In programming may not be very 
advanced, Frequently, It consists of undergraduate computer or programming 
courses, supplemented by on-the-job training and an occasional extension course. 
In some groups at least one Individual has a degree In computer science or a 
related fleld. The motivation of Individual programmers Is governed by the 
needs of thelr application. In the simulation fleld, which represents a 
signif.icant part of the smaller sclentific programming community, the code tends 
to be bound to a specific facility. Although most programming Is In FORTRAN, 
there may be frequent recourse to assembly routines to speed up the execution. 
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As a rule, structured programming Is not used “In that. envi ronment al shaugh 
Individual programmers may be experimenting with It. 


Eng! neer!ng and sclentific analysis progr és are sometimes distributed outside 
the originating organization. and In those cases portability Is a recognized . 
requirement, at times enforced by a portability analyzer. Structured and 
modular’ programming Is’ used more frequently than In the simulation fleld but /Is 
seldom formally required, Another characteristic of the eng! neering or 
sclentific analysis environment that affects too! usage Is that many programming 
tasks are of less than 3 months duration, and much of that time Is spent on 
analysis of the underlying problem rather than on program design or 
Implementation, This discourages the use of tools that require much set-up i 
or a lengthy learning perjod. 


The emphasis In the sclentific programming envlronnent Is more on the generation 
of new programs as contrasted with malntenance. Some malntenance activities, 
such as the addition of ajmajor feature to a simulation, or the extension of the 
capabilities of an analysis program, are regarded as creative and desirable 
assignments. More typ!ical maintenance activities, such as modifying a report 
format, adapting a program to a change In hardware configuration, or correcting 
Interface problems are regarded as less desirable assignments and.are glven to 
Juntor personne! as "tralning", 
$ 

Supervisors regard: the documentation and detalled maintenance as problem areas. 
Although these are recognized as essential elements of the organization's 
overall assignment, the senior programmers take |Ittle Interest In them, and a 
good methodology Is not avallable for breaking them down Into tasks that could 
be efficiently handled by\less experlenced personnel or by personnel not 
Involved In the direct programming. Some smaller organizations make effective 
use of general purpose development tools to strip headers and comments from 
programs and to transform these Into documentation. More typical Is the 
approach where supervisors, In some cases second level supervisors, assume the 
major respons {bil Ity for the review of documentation. 


A very significant part of the overal| activity In the simulation fleld Is 
version control, New assignments frequently consist of assembl Ing existing 
modules, ‘some with minor changes, Into a new configuration, File management: 
systems can be used very effectively to assI'st In this process. Editors and 
-preprocessors (usually without extensive analysis features) are other typical 
tools currently used In this environment. 


By and large the sclentific programming organizations have'the technical ability 
to acquire and Install software tools. They may lack specific Information on 
tools suitable for thelr environment, the resources for the Introduction, and 
frequently also the motivation to devote part of thelr effort to software 
engineering. Because of the recognized difficulties In documentation and 
malntenance, the second level supervisors will be particularly receptive to 
tools that can simplify the work In these areas. ‘ 
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This section discusses those aspects of user tool needs that are pertinent to 
the development of guidance for the Introduction of tools. Subsection 4.1 
considers organizational factors of tool needs that are largely Independent of 
the application area. This‘is followed In subsection 4.2 by a detalled 
Identification of tool features desired [n‘the target environments described In 
Section 3. Even experlenced tool users can be faced with severe problems In the 
adoption of new tools, and the needs that arise In this connection are addressed 
In subsection 4.3. The final part of this section describes resources available 
to the potential tool user for selection of specific tools to meet the needs 
characterized In the earlier parts. 


‘4.1 ORGANIZATIONAL FACTORS IN TOOL NEEDS 


. The objectives of toof usage (and hence the objectives of many tool features) 
are determined largely by the user's organlzational environment and by the 
management level that authorizes tool acquisitions, Because these 
considerations hold for all application areas, “they are discussed af the 
beginning of this section. For a tool to be readily accepted. It must help In 
areas. of concern to the, management that authorizes the acquisition and 
Introduction activities. Thus, If management considers program documentation to 
be a particularly critical area It may be difficult to obtain authorization for 
the Introduction of test tools. The organizational entities that may be 
Involved In the acquisition and use of software tools aradescribed under the 
headings of: 


Software Devel opment Organization, 
Project Management, and 
Functional Management. 


4.1.1 Software Development Organization 7 
a 
The term development Is used In a broad sense that Includes al! the activities 


directly Involved In generating and malntalning programs. Practically all 
software development organizations desire tools that: 


Increase productivity. 

Reduce skIiIl requirements. 

Automate routine aspects of software design. 
Help In software maintenance. : 


To some extent the last three Items are Individual facets of the first. At the 
present time there are few tools that are specifically almed at a reduction of 
skill requirements. The creative and cognitive skills required for designing a 
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sound software structure are not easlly packaged Into a ‘software tool. However, 
the automation of routine tasks Js a very widely addressed tool objective. 
Because they relleve creative personnel from tedious aspects of design, coding, 
and testing, these tools compensate partly for the lack of those that reduce the 
skIl| levels. They also contribute to Increased productivity. Examples of such 
tools are editors and precompllers used for the preparation or conversion of 
source filles, formatters for the preparation of reports, and sort/merge 
programs. The most common tool features that automate routine tasks are 
editing, formatting, comparison, translating, and scanning. Tools that help In 
‘software maintenance Include most of those clted for the atttomation of routine 
tasks plus file managers or IIbrary systems. In addttion, maintenance may make— 
use of special functions In editing or scanning tools, e. .g., to locate 
variables or to strip code from a source file. The latter feature Is useful for 
creating documentation from the Fregres comments. 


All of the tool functlons and features enumerated here are of direct banet tt to 
the software professional, and there Is seldom any difficulty In Introducing 
them at the working level If they provide a reasonably friendly user Interface. 
Line management In the software development organization may need ‘to be 
convinced that the cost of the tool acquisition and Introduction will be 
recovered over a reasonably short time span. Note that the use of these tools 
Is largely Independent of emphasis on standards that’ may prevail In the using 
organization. . 


Where standards are In use, additional tools will be desired that either 
facilitate or enforce compliance. Among the former are program design language 
processors, and among the latter are code auditors. Environments that emphasize 
standards usually also demand discipline In the procedural aspects of software 
development such as version control, access to test cases, etc. File managers 
and |ibrary systems will be found helpful In enforcing this discipline. Tools 
that support standards will be readily accepted by a standards-minded IIne 
management. The professionals who have to use the tools may regard them with 
Indifference or even hostility. Part of this Is due to apprehension about 
having one's work scrutinized by "Big Brother", arid part Is due to obstacles to. 
Innovation (deviation from standards) which these tools may present. It Is 
therefore Important that tools of this type have a particularly good user 
Interface so that potential complaints about thelr use can be minimized. Some 
tools, such as auditors, can be combined with the compliler so that they are 
automatically ‘Invoked when a new source file Is submitted. This Integration 
makes more efficient use of the computer and at the same time avoids problems at 
the user Interface. 5, 


That they support or enforce standards Is a particularly pertinent factor In 
connection with the Introduction of software tools to Government agencles. 
Beyond the benefits that always attend uniformity of design practices. 
Government agencies will find It easier to Interchange both programs and 
personnel! If common standards can be adopted. Some of these benefits transcend 
the usual concerns of the soft®are development organization. The broader 
aspects of tool usage to support software standards are discussed under 
Functional Management In 4.1.3. 
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Because Government agencles can have access to tools developed or In use by 
other Government organizations, they may particularly benefit from the 
appoIntment of a local toolsm|]th -- a person expert In tool usage who may be 
able to make software or,minor hardware modifications that perm\t a tool to be 
used In a new environment. The role of the toolsmith was Introduced several 
ears ago as a specialist within a software development team In these words 
BROOTS 1: 


(Thé team teader) needs a toolsmith, responsible for ensuring this adequacy 
of the basic service and for constructing, maintaining, and upgrading 
special tools -- mostly Interactive computer services -= needea by his 
team. Each team will need Its own\toolsmith, regardiess of the excellence 
and rellabllIlty of any centrally proVided service, for hls Job Is to see to 
the tools needed or wanted by his (team). without regard to any other 
team's needs. The tool-bullder will often construct specialized utilities, 
catalogued procedures, and macro |Ibrarles. 


The designation of a dedicated toolsmith within each team may be a higher degree 
of specialization than can be warranted In smaller ‘software organizations. 
However, within each software environment that makes use of a single computing 
facility such a specialist will be found very effective and certainly very 
valuable for the Introduction of new tools. 

i 


4.1.2 Project Management 


Project management directs the software development on behalf of the ultimate 
user. It Is usually more Interested In the functional and Interface aspects of 
the. programs than In structural or standards aspects. Where project management 
Is funding the acquisition of software tools, there may be heavy emphasIs on 
tools that have an Immediate payoff In terms of project objectives. Some of 
these tools may be software development tools but the nature of these Is project 
dependent and can not be predicted. 
There are, however, some. software tools that make a direct contribution to 
project management, and this area of tools usage Is expected to be expanded In 
the near future. Some programs of this kind are general purpose schedu | Ing and 
reporting algorithms that share more of the characteristics of application 
programs than those of software tools. Others. however, are very specific to 
the software area and extract Information from the software as It Is being 
developed, These are appropriately described as software tools for project 
management and are further described below. 


Software | Ibrary systems have already been mentioned In the’ previous heading as 
tools that can support discIpiined development and ald In software mal.ntenance. 
For project management they can furnish the Identification and date of the 
latest revision, current file size (number of statements), and change In size 
over a selected time Interval. Elther by themselves or In conjunction with the 
operating system log, these tools can also‘furnish reports on the total number 
of rufis, the number of statement changes, the number of different test cases 
submitted, and the number of compilation fallutes or aborted runs. All of this - 
Information can be furnished In hard copy or Interactively on a terminal. In 
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elther format, tools furnish thesé data more convenlently and at a fraction of 
the cost of manual methods. 


Tools for cost estimation are also Important for the project management area. 
Development cost, IIlfe cycle cost. and computer cost aspects can be estimated by 
means of software tools. Development costs are estimated by automating an 
estimation algorithm such as [DOTY77]. Life ¢ycle costs can be developed as an 
extension of the development costs, such as In the ESD model [JAME77], or from 
data on a system under development such as [PUTN78]. Computer cost aspects are 
estimated by sizing and timing tools. While the jury Is still out on the 
accuracy of the cost estimates generated by these tools at present, there Is 
little doubt that thelr use promotes systematic collection of software cost data 
and a methodical approach to software costing. 


Since the emphasis In thls report Is on theylntroduction of tools to smaller 
_ programming environments, I+ should be noted that not all project management 
_tools need to be vegy large systems. Management will frequently derive 
considerable benefits from small programs that automate follow-up on action 
Items, recelpt of dellverables from vendors, etc. Programs of this type can be 
applied In any environment regardless of size. 


~ 


4.1.3 Functional Management =. ° 


In organizations where software Is belng developed for more than one project, 
the Individual development groups usually report to a common management level 
which Is referred to here as the functional management or computing function 
management,- Because personnel! must be periodically reassigned to new projects, 
functional management will usually be Interested In uniformity of practices 
among projects so that retraining can be minimized. . Computing function 
management can thus be expected to be standards-orlented and to support the 
Introduction of tools that enforce standards. This management level Is usually 
Inclined to take a long range view and may favor the acquisition of tools that 
primarily benefit later software | ifecycle phases.:e. g., requirements analyzers 
(although these are used during the definition stage, the major benefits are 
usually reduced maintenance costs during the operation phase). 


Functional management Is usually also Involved In another Important aspect of 
the Introduction of software tools: It must furn or allocate the facilities 
for the execution of the tools. Very few compyting facilities have excess 
capacity. and this Is particularly true for Government computing facilities. 
Therefore, the management of the computing function may object to the 
Introduction of tools that extend the execution time or that add job steps to 
frequently run programs. Where a tool has a significant adverse Impact on 
throughput, the benefits of that tool In areas of concern to functional 
management should be highlighted: Increased programmer productivity, adherence 
to standards, or Improved software qual ity. 
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Because tool Integratton avoids repeated reformatting and multiple data 
retrievals, It reduces computer usage and supports the goals of functional 
management, It Is at this management level that the greatest recognition of the 
nener tts of tool Integration ef forts can be expected. 


Fanetlonal management will also be (uterested In tools useful for the management 
of the computing factlity, e. g., those that allocate charges to users. that 
report on the operation of the current facllities, and simulation tools that ald 
In planning of Improvements. The features of Instrumentation, resource 
utilization, simulation, and statistical analysis support the capabllities of 
such tools. ‘ 


4,2 APPLICATION FACTORS IN TOOL NEEDS 


The following discussion focuses on the needs of the two environments that were 
Identif ted In Section 3 as the primary targets for the Introduction of software 
tools: the smaller MIS organization and the smaller sclentific programming 
organization. In the studies leading to the definition of tool needs, sIx 
application areas were considered: business-orlented batch systems. management . 
Information systems, offlae automation systems, online transaction driven 
systems, real time command and control systems, and sclentific or engineering 
programs, It was found that the tool needs of the first four of these were very 
similar, and this entire group Is encompasséd by- “the discussion In 4.2.1 below. 
Also, the software tool needs of the last two categdries were Identical, and 
these are described In 4.2.2 below. Within this Subsection It Is assumed that 
the tool types and features required for ths ‘general software deve! opment 
organization [4.1.1 above] are provided, and therefore only the supplements 
dictated by the speclific application areas are discussed. 


4.2.1 Tool Needs of Smaller MIS Organizations : F 

One of the distinctive tool needs of this environment arises from the use of 
COBOL and the Inefficilencies of COBOL compilers that were mentioned earl ler 
[3.3]. A sizeable number of commercial tools have been developed to Improve the 
performance of COBOL programs. Two specific techniques have been found 
particularly helpful In this area: modifying the object code for Improved 
performance (the significant tool feature for this Is optimization), and 
determining and simplifying the parts of the program which account for the bulk 
of the run time (the significant tool feature for this Is tuning). Of course 
both of these can also be used together. Tuning Is part of the dynamic analysis 
function. It generally requires Instrumenting the program, |. e., the Insertion 
of code that counts the number of accesses to the program segments of Interest. 
Once this Is done, other attributes of the program's structure and performance 
can also be evaluated, and such options are provided In several of the 
optimization tools. In connection with the Introduction of tools Into the 
smaller MIS organization, It Is suggested to avold such additional capabilities 
In the tool Initially because they extend the run-time of the instrumented 
program, and they make the user Interface more complex than It needs to be. 


’ 


ee 


>» Tools can be used very effectively to ald In program conversion (e. g.- when a 
new computer Is belng Installed) which can present many problems In the smaller 
MIS environment., Over 270 conversion tools are IIsted hn a publication of the 
Federal Conversion Support Center [FCSC80]. The IIsting Includes tools that 
facilitate conversion (e. g., translators), as well as programs that may 
eliminate the need for conversion (e. g., emulators). 


Because of the heavy Involvement of the MIS applications area In the 
7 manipulation of data structures. tools that simplify data base updating and 
restructuring are another specific need of this environment. While program 
libraries and general purpose fille management systems mentioned In 4.1.1 can be 
of some help for updating, specialized systems for data base management are 
preferred. A number of these are commerclally avallable, and they frequently 
combIne access control, archiving (or providing an audit trafl), auditing for 
completeness and reasonableness of the Inserted data, and restructuring with the 
update capabliity. Data encryption Is offered as an optional feature of some 
tools but Is not considered essentfal for the smaller MIS environment. ; 


| 
4.2.2 Tool Needs of the Smaller Scientific Programming Environment 


. Just as the use of COBOL Is responsible for some specific tool needs In the MIS 
‘environment, the use of FORTRAN, the current leading language In the scientific 
programming environment, has In the past also been a strong motivator for 
specific tools, In this case pre-processors for structured languages. 
5 A cml frequently represent the most advanced software tool In the 
Inventory of smailier sclentific programming organizations. Even though 
pre-processors will continue to be used, It Is suggested that a forward looking 
program for the Introduction of tools to the smaller’ scientific programming 
environment not emphasize this tool type unduly. One of the reasons Is that, In 
sclentific programming for the defense-connected sector, FORTRAN is belng 
replaced by languages whIch Inherently support structured programming practices, 
and another reason Is that sultable control constructs are evolving for FORTRAN 
CANSI78]. But more fundamental Is the need to educate the sclentific programmer 
In the smaller organization to the benefits of a methodical approach to program 
development of which structured programming Is but a part. 


This need can be met by a general purpose development tool package that takes 
the drudgery out of some of the routine programming steps. One such package, 
described In the professional |Iterature [KERN76]. Is In the public domain, 
This approach, which Is clted only as an example. Involves a number of 
Independent utilities that can be Invoked Individually or Interactively by means 
of a command IIne processor to do editing, file management, formatting, and 
pre-processing. The efficient application of the tools poses an Intellectual 
challenge’ that may be particularly motivating for sclentific and engl neering 
personne! who are the progr rs In this environment. Software tool packages 
patterned along these IIneS are In use In some smaller sclentific environments, 
and the user reaction seems to be favorable. 


nN a 7 : 18 
Once a sclentific programming organization.Is committed to a disciplined 
development approach (and as previously explained some of the smaller 
organizations are already at that point), needs for many static analysts 
features may arise. Including code auditing, completeness checking. consIstency 
checking, error checking, and statistical analysls., Tools with these 
capabliities will be particularly desired for design analysIs programs 
supporting critical applications (nuclear Industry, alrcraft structures) and for 
simulations which furnish output to physically active equipment (e. g., moving 
base flight simulators). At present, smaller organizations may find It 
difficult or Impossible to acquire these tools In a useful format. Once a 
general purpose development tools package Is In use, the additional checking 
tools can be developed In-house. Commercial sources may become Interested In 
adding checking functions to an established basic tools package. 


Real-time command and control systems pose additional requirements that may 
require dynamic analysis features. At.present, this applications area Is 
primarily served by large organizations that make extensive use of tools. the 


4.2.3 Summary of Too! Features Determined by the Appi ication 


Table 4-1 IIsts common and applicatlon-dependent tool features. The first 
column |Ists features desTred~by all software development organizations as 
described In 4.1.1. The features shown In the table are the ones most desired 
by new tool users. The selection Involved some Judgment regarding the 
. priorities that exist within the software development organization and Its user 
community. Thus, that error checking Is found In the sclentific column but not 
In the MIS column does not Imply that this feature Is not sultable or not 
Important In the MIS environment. It doBs mean that most smaller MIS developers 
ne A pats a lower priority on error checking than on the features II!sted In the 
S$ column, 


TABLE 4 - 1 FEATURES DETERMINED BY THE APPLICATION 


Features Needed For 


_ Common MIS Programming Sclentific Programming 
Editing Optimization Auditing (Code) 
Scanning _ Tuntng Completeness Checking 
Formatting Restructuring Statistical Analysis 
Comparison Auditing (Data) Error Checking 
Translation Consistency Checking 
Management 
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4.3 NEEDS OF OTHER ENVIRONMENTS 


Although large and very large software organizations are In most cases already 
users of general purpose software tools, they may benefit from programs almed 
at Improving access to tools, standardization of tool Interfaces. or 
establishing minimum requirements for tool documentation and diagnostics. 
Because these organizations (which will be collectively called "larger") have 
multiple general purpose tools In Inventory, the Integration of tools Is 
particularly pertinent for them. There are at present no clear Indications of 
the direction that tool Integration should take. However, there Is a 
considerable effort belng devoted to the area of programming environments within 
NBS and elsewhere, and tool Integration Is an Important aspect of these 
activities [BRAN81, IEEE81]. 


The Integration of tools provides two primary benefits: a simplified user 
Interface and reduced util Ization of computer resources. The simplified user 
Interface Is achleved by requiring less file manipulation for converting the 
output of one tool Into an Input for another, by consistent tool calling 
conventions, Input commands, and output formats, and by the capabllIity for 
Invoking processing by several tools with a single command string These 
benefits wlll In turn simplify documentation and training and In general Improve 
the user acceptance of a system of multiple tools. 


The reduced utIlization of computer resources Is partly due to the factors just 
enumerated, particularly the avoidance of file manipulations, and partly due to 
the possibility. of combfning computer-Intenslve operations such as parsing and 
searching which are now carried out separately. As mentioned In 4.1.3, the 
managers of the computing function will particularly appreciate these latter 
benefits. The reduction In running time and storage requirements together with 
the benefits due to the simplified user Interface promise a high Beyer for 
efforts In the tool Integration area. 


A significant step for the Integration of tools developed by different sources 
Is represented by the NBS/ICST study of compIler-based tools [BRAY81J. The 
assoclation of the tool with the compiler provides access to at least two 
(source and object) and sometimes three (parsed code) representations of the 
program, and also makes the data and structure checking features of the compIler 
avallable to the tool. A possible disadvantage of thls approach Is that a 
compiler pass may be required In order to Invoke a too]. The adherence to a 
single (or at least compatible) file format for multiple tools can be readily 
enforced by compller basing. 


? 
The Integration of tools Is very significant for extending tool usage In 
environments where general purpose tools are already In use. It Is of lesser 
Importance for the introduction of tools to environments that had no prior 
expertence with general purpose software tools, and It Is therefore addressed 
here In only a IImited way. 


A topic partly contained within the area of tool Integration Is the 
standardization of tool commands and output formats. The lack of 
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) standardization is particularly obvious and disadvantageous’ In editors and 

; related tools (Including word processors). These are among the most widely used 
tools, they are frequently the medium through which the Input to other tools Is 
processed, and there Is better agreement than In most other areas on the 
functions which the tool Is required to furnish, There Is thus ample motivation 
to standardize but very few concrete accomp!Ishments. 


There are at present many different methods for cursor positioning, different 

commands for deleting characters, words, or lines, and different procedures as 

well as commands for string search or substitution. These InconsIstencles cause 

errors, necessitate multiple training periods, and certainly constitute a 

deterrent to tool usage. In view of the basic need for an editor In the use of 

many tools, the lack of standardization of editor Pemmanes flust be regarded as 
* an obstacle to the Introduction of tools. 


4.4 RESOURCES FOR TOOL SELECTION \ 


In subsections 4.1 and 4.2 a number of generic tool types and features ‘have been 
Identifled as sultable for the Introduction of tools to specified organization 
and application environments. The present subsection discusses the additional 
‘steps necessary for the actual selection of a tool. ‘ 


Gatalogues of software taols are avallable from the commercial tool developers, © 
computer manufacturers, and computer users' groups. For obvious reasons, the 
offerings In each of these are restricted although the restrictlon Imposed by 
- the last of these may be an appropriate one If only the computer type addressed 
"by that users' group ‘Is avallable as a tool host and If the ed has conducted ~ 
a comprehensive survey of sultable tools. 


“A recent publication by the National Bureau of Standards Is particutarly helpful 
for the Introduction of tools to smaller programming environments [HOUG82]. I+ 
contains cross references by tool classification (function) and features which 
makes It especially sultable for use with the tool Ident! fications used In this 
eepeens ‘ : 

Once a tool that meets the functional requirements Is Identl fled, the main 
section of the catalogue must be consulted to see whether the tool Is usable on 
an avallable computer, whether It handles an appropriate source language (or |, 
other Input format), and whether It can be obtained In a suitable ‘Implementation / 
language. Other considerations are the IIcénsing arrangements, avallabl lity of / 
documentation and aaa and the computer resource utIl| ization. | 
Some difficulties are usua | ly encountered that must be resolved by language 
conversfons (of elther Input or the tool) or other tool modIfications.- 
Consultation with a toolsmith wil! be valuable In this connection. Government 
agencles will want to know whether other Agencies are currently using the tool, 
and a central tool usage catalogue wil! be a beneficial facllity.. Access at a 


“ remote computer can be a very effective first step In a detailed evaluation of 
the tool. ‘Hosting ee may be overcome by remote access even on a longer 
term basIs. 

) 
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’ SECTION 5 


DEVELOPMENT OF EVENT SEQUENCES 


While the preceding sectfons have discussed the tool needs In selected 
environments, this section describes the detalled events that will lead to 
suocessful use of tools. The first subsection describes the purpose and 
rationale for an event sequence. and the second subsection recommends a specific 
event sequence for the smaller MIS and sclentific environments. 


5.1. PURPOSE OF EVENT SEQUENCES . 


The management of any significant project requires that the work be divided Into 
tasks for'which completion criterla can be defined. The transition from one 
task to another Is called an event, and to permit orderly progress of the 
activities, here the Introduction of a software tool, the scheduling of these 
events must be determined In advance. A general outiine for such a schedule Is 
provided by the event sequence described In the next subsection. The actual 
calendar time schedule will depend on many factors which must be determined for 
each specific tool use (particularly on the time required for procurement of the 
tool and training). . One of the formats used for the event sequence Is 
consistent with the Critical Path Method (CPM) of project schedullag and can be 
=e rita that ‘technique for the development of an optimum calendar time 
schedule, 


Most of the activities Included In the event sequence are obviously necessary 
but a few were Included specifically to avold difficulties encountered In 
prevlous tool procurements. Quite frequently tools were obtalned 'through the 
Side door' without adequate consideration of the resources required for the 
. effective employment of the tool and without determination by a responsible 
manager that the tool served a primary need of the organization. Tools acquired 
In this manner were seldom used In an optimal way and were sometimes discarded. 
Experiences of this type are not conducive to galning widespread acceptance of 
too}s In the smaller programming environments where the activities required for 
the Introduction of tools wlll, under the best of circumstances, -Impose a severe 
draln on resources. A key feature of the proposed approach Is, therefore, that 
tool usage will be Initiated only in response to an expressed management goal 
for software development or for the entire computing function. 


, alas In the Introduction of tools, can arlse In three areas: 
Organizattonal obstacles 


- Problems arising from the tools 
Obstacles In the computer envi ronment 


The Individual activ ties described below as well as the ordering of tthe event 

_sequence are designed to eliminate-as many of these difficulties as possible. 

“ They are most effective with regard to the first category and probably least 

effective with regard to the last category. The need ‘for Involving a reponsible. 
: \ ee 
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management level tn the tool Introduction has already been mentioned, and this 
Is Indeed the key provision for avolding organizational obstacles. "Responsible 
management" |s that level that has the authority to obligate the resources 
required for the Introduction process. The scope of the resource requirement 
will become clearer after all Introduction activities have been described. 
Because the criterion for the selection of the management focus Is Its abillty 
to commit. funds, this management level is hereafter referred to as funding 
management. In some organizations this may be the project management as def Ined 
In 4.1.2, In some It may.be functional management as defined In 4.1.3, and In 
yet others It may be an agency or department management not specifically 
Identified with a computing function. “It should be Involved In at least the 
following activities assoclated with the ip roduetion of tools: 


1. Identifying the goals to be met by fhe tool (or by the technIqué supported 
by the tool), and assigning responsibIlIity for the activities a al to 
meat these goals. 


2. aanbouing a detailed too! acquisition plan that defines the resource 
requirements for procurement and ‘In-house . activities. - 


3. Approving the procurement of fools and training ‘If this Is not explicit In 
- the approval of nae acquisition plan. 


a, 


4. Determining after some period of tool use whether the goals have been met, 


Additional organizational obstacles must be ovércome by actions of the software 
management (local management of the organization that wlll Introduce the tool). 
A pitfall that must be avoided 1s assigning the detalls of the tool acquisition 
as a sideline to an Individual who carries many other responsbilItles. Even In 
a small software organization ‘(up to 14 programmers), It should be possible to 
make the tool Introduction the principal assignment of an experienced Individual 
with adequate professional background. This person Is. referred to as the 
software engineer. In medium size organizations (15 to 39 programmers) several 
Individuals may be Involved In software engineering tasks: (not restricted to 

tool usage), and this may constitute a software engtnger ing function. . 


Further, the event sequence Includes activities of a toolsmith who witi not be © 
the same person as the software engineer In most cases. The former. assignment 
requires expertise In systems programming and speclaltzed knowledge of the tool 
_ to be Introduced, The dutiés of the software engineer Involve planning project 
management, and obtaining cooperation from a variety of Individuals and 
organizations. Where there Is a software engineering function, ihe toolsmith Is 
typically a member of It. : 


‘ Obstacles, arising from the tools themselves are expected to be avolded In the 
event sequence by a careful, methodical selection of tools, In particular 
distinct contributions to the tool selection are specified for software 
management and the. software engineer, Software management Is assigned 
responsibility for: ; . 
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_ Ident fying tool objectives. 


Approving the acquisition plan (It. may also require 
approval by funding management). 


- Defining sefection criterha. eo 


-MakIng the final selection of the tool or the source. 


The software engineer Is responsible for:  * 


Identifying candidate tools. 


Applytng the selection criteria (In Informal procurement) 
or preparing RFP Inputs: (In formal procurement). 


Preparing a ranked IIst of tools or sources. 


Further, the ultimate user of the tool Is Involved In the recommended event 
sequence In reviewing efther the IIst of candidate tools or, for formal 
procurement, the tool requirements. 


‘ This Alstribution of. responsibilities reduces the aiaheee of selecting a tool 
that (1) does not meet the recognized needs of the organization, (2) Is 
difficult to use. (3) requires excessive computer resources, or (4) lacks 
adequate documentation. The repeated exchange of Information required by the 
process outlined above will also avold undue emphasis on very short-term 
objectives which may lead to selection of a tool.on the basis of availability 
rather than suitability. 


The obstacles to tool usage that reside In the computer environment are 
primarily due to. the great diversity of-computer architectures and operating 
system procedures, and to the lack of portability In most software tools. 
Activities assoclated with the Introduction of tools'can only modestly alleviate 
these difficulties. The event sequence: provides the following help In this 
area: ‘ 


1. A methodical process of Identifying. candidate tools and selecting among 
these on the basis of established criterla. This will avold some of the 
worst pitfalls associated with "borrowing" a tool from an acquaintance. or 
procuring one from the most accessible or persuasive tool vendor, 


2. The assignment and training of a toolsmith who can make minor modifications » 
to both the computer environment and the tool. This Is expected to provide . 
rellef where there are verslon-related or release-related Incompatibilities 
‘with the operating system, or where the memory requirements of the tool 
exceed the capabilities of the Installation. In the latter case, remedies 
may be .provided by removing tool options or by structuring the tool program 
Into overlays. 


? 
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The event sequence described below Is concelved as a procedure general ly 
applicable to the Introduction of tools to Federal agencles falling Into 
pertinent programming environment categories. For this reason, a systematic 
reporting of the experience with the Introduction process as well as with the 
tool Is desirable. The evaluation plan and. ThE evaluation report specified In 
the event sequence support these goals. 


« 
5.2 RECOMMENDED EVENT SEQUENCE 

The event Seneca described In this subsection Is applicable to both the 
smaller MIS and scientific programming environments. The general scope of the 
Introduction activities and thelr sequence are Identical for the two 
environments. Because of differences In tool requirements, personnel 


qualifications, and organizational structure, some differences In the.content of © 


the IndIvidual events will be expected. The event sequence addresses only the 
Introduction of existing tools. Where a newly developed, tool Is Introduced, a 
considerable modification of the activities and thelr sequence will be 
necessary. 


The recommended event sequence akd@ws for two procurement methods: Informal 
procurement (e. g., by purchase order) or formal procurement by request for 
bids. Obviously, the latter’ Is much more time*consuming but It may lead to ‘the 
procurement of better or cheaper tools. Acquisition of tools from the General 
Services Administration or from.other Government agencles should follow the 
Informal procurement steps even when there’Is no procedural requirement for 
this. As mentioned above, tool acquisitions which do not obtain -the concurrence 
of all affected opersriones elements frequently do not achleve thelr objectives. 


The presentation of the event sequence In Table 5-1 Is tallored to tools which 
are belng Ihtroduced for the first time Into a user community which shares 
software support Information (e. g., a Federal agency or a private sector 
company). As a result, some steps are shown which can be combined or eliminated 
where less formal control Is exercised or where plans or modifications required 
for the ‘introduction of a tool are avallabl6é from a prior user. The event 
sequence Is Intended to.cover a wide range of applications, and It was 
constructed with the thought that It is easier for the tool user eliminate 
steps than to be confronted with the ney for adding some tha nes not been 
covered In this volume. 


The key functions which contribute 6 the Introduztion of tools are Jisted 
across the top of Table 5-1, and events for which each function Is responsible 
are IIlsted “In the column under It. The preferred order of tasks for each 
_ function can thus be directly found from this table. e@ precedence 
relationships between events 1s shown In graph form In Figure 5-1. This figure 
will be found particularly helpful for scheduling activities by the Critical 
Path Method-.and. for the general development of a project schedule. The 
numbering of events Is the same In Table 5-1 and Figure 5-1. A detalled 
description: of each of the numbered events, and of the activities assoclated 
with It, Is presented following the table and figure. 


BU 
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P a 
fit * TABLE 5 - 1 EVENT SEQUENCE FOR TOOL INTROOUCT ION 


FUNDING SOFTWARE SOFTWARE TOOL 
MANAGEMENT ‘MANAGEMENT ENGINEER : USER 


1. Goals 
2. Tool Objectives 


A <eeenem---r= 4, Evaluation plan 
A <enmeenener= 5. Toolsmitilpg plan- 
fj; Seeweoennte G6. Training plan <<< 


3, Procure tool 


participates 


7. Recelve tool 
8. Acceptance test 


9. Orlentation 


10. Modif ications-S 
Keomeeee= 11. Training sesns--= > 
12. Use 


Do Rematicekiee IA eimmduomeieons 13. Evaluation report 


Goals met? 


14, 


Al. Acquisition plan 
A2. Select'n criteria 


A3. Ident. candidates 
A5. Score candidates 


A4. Review 


A6. Select tool 


B2. Technical req'mts 
B4. Generate RFP 
B6. Proposal Evaluation 


B3. Review 


B7.:Select source 


continue with step 3 above, 


A= Approval’ required : S$ = Toolsmith responsIbIIIty_ 


EVENTS 


1. Goals 

2. Tool objectives 
3. Procure tool 

4. Evaluation plan 
5. Toolsmithing plan 
6. Training plan 
7.,Recelve tool 

8. Acceptance test 
9. Orlentation 

10. Modifciatijons 

11. Training 

12. Use 

13. Evaluation Report at 
14, Goals met? 


Al. Acquisition plan 
A2. Selection criteria 
A3. Identify candidates 
A4. User review 


A5. Score candidates 
B. FORMAL PROCUREMENT AB. Gel sck deel 


~ A. INFORMAL PROCUREMENT 


pv a 


BI. Acquisition plan 

B2. Technical req'mts 

B3. User review 

B4. Generate RFP 

B5. Issue RFP * 

_B6. Proposal evaluation — 

B7. Select source \- 


32 _ FIGURE 5 ~ 1 PRECEDENCE RELATION FOR EVENT SEQUENCE B. 
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1. Goals 


The goals to be accomplished should be Identified In a format that permits later 
determination (event 14) that they have been met. Typical goal statements are: 
reduce average processing time of COBOL programs by one-fifth; achleve complete 
Interchangeabil ity of programs or data sets with organization Ys adhere to an 
established standard for documentation format. : 8 


The statement of goals shall also Identify responsibilities, In particular the 
role that headquarters staff organization may have and coordination requirements 
with other organizations. Where a decentrallzed management method Is employed. 
the statement of goals may have assoclated with It a not-to-exceed budget and a 
desired completion dats. Once these constraints are specifled. funding 
management may delegate The approval of the acquisition plan to 4 lower level. 


2. Joo) Objectives 


The goals generated In event 1 are translated Into desired tool features (¢e.g., 
see Table 4-1), and requirements arising from the development and operating 
environment are Identified. Constraints on tool cost and availability may also 
be added at this event. A typical:statement of tool objectives for a program 
formatter Is: Provide header’ Ident[fication, uniform Indentation, and the 
facility of printing IIsting and comments separately for all FORTRAN X3.9-1978 
and ABC Extended FORTRAN programs. Program must run on our ABC computer under 
XOSnn. Only tools which have been In commercial yse for at least 1 year and at 
no less than N different sites shall be considered. 


At this point the sequence continues with elther Al or BI below. 


The acquisition plan communicates the actions of software management both upward 
and downward. The plan may also be combined with the statement of the tool 
objectives (event 2). The acquisition. plan should Include the budgets and 
schedules;for subsequent steps In thé tool Introduction, a Justification of 
resource Peqllressits In the light of expected benefits, contributions to tye, 
Introduction expected from other organizations (e. g., the tool Itself, 
modification patches, or training materials), and the assignment of 
responsibility for subsequent events within the software organization, - 
particularly the Identification of the software engineer. Minimum tool 
documentation requirements shall also be specified In the plan. , 


A2. Selection Criteria 


The criteria shall Include a ranked or welghted ilsting of attributes that will 
support effective utilization of the tool me the user. Typical selection 


criteria are: | x 


NS 
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Accomplishment of specified tool objectives. 
Ease of use, ’ 
Ease of Installation. 
Minimum processing time. 
Compatibility with other tools. 
_ low purchase or lease cost. 


Most of these criteria need to be factored further to permit objective 
evaluation, but this step may be left up,to the Individual who does the scoring. 
Togethér with the criteria (most of which will normally be capable of a scalar 
evaluation)s constraints which have been Imposed by the preceding events or are 
generated at this step should be summarized. 

AS. ldentify Candidate Tools 

This Is the first event for which the software engineer Is responsible. The 
starting point for preparing a listing of candidate tools Is a comprehensive 
tool catalogue, such as [HOUG82]. .A desirable but not mandatory practice Is to 
prepare two IIists, the first of which does not consider the constralnts and 
contains all tools meeting the functional requirements. The Cross-Reference by 
tool features In the appendices of [HOUG82] will be found particularly valuable 
In generating this IIst of candidates... For. the example used In event 2, a 
program formatting tool, 16 entries are found there. Some of these may be 
eliminated by further review of thelr description In the body of the catalogue 
(e. g., because they don't process the specified FORTRAN dialects). For the 
remaining viable candidates, Iiterature should be requested from the developer, 
and this Is examined for conformance with the glven constraints. At this polnt 
a second list Is generated, containing tools that meet both the functional 
requirements and the constraints. If this IIist does not have an adequate number 
of entries, relaxation of some constraints will have to be considered. 


A4. User Review of Candidates 
The user reviews the IIst of candidate tools prepared by the software eng! neer. 
Because few users can be expected to be very knowledgeable in the software tools 
area, specific questions may need to be ralsed by software management such as: 
"WIL 1 this tool handle the present fle format? Are tool commands censistent 
with those of the editor? How much training will be required?" Adequate time 
should be budgeted for this review le due date for responses should be 
- Indicated. Because the useg views this as a far-term task, of lower priority 
than many .Immediate obligations, considerable follow-up by IIne management will 


be required. If tools can be obtained for trial use, or If a demonstration at 
another facliIty can be arranged, It will make this step much more significant. 


A5. Score Candidates 
‘ 


For each of the criterla previously Identified a numerical score Is generated 
on the basis of Information obtalned from vendor's IIterature, from. 
demonstration-of the tool, from the user's review, from observation In a working 
environment, or from comments of prior users. If welghting factors for the 
criterla are specified, the score for each criterion Is multipiied by the 
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appropriate factor and the sum of the products represents the overall tool 
score. Where only a ranking of the criteria was provided, the outcome of the 
scoring may be simply a ranking of each candidate under each of the criteria 
headings. Frequently a single tool Is recognized as clearly superior In this 
g Drewes : : 


A6. Select Tool 


This decision Is reserved for software management In order to provide review of 
the scoring, and also to permit additional factors which were not expressed In 
the criteria .to be taken Into consideration. For example, a report might just 
have been recelved from another agency that the selected vendor did not provide 
adequate service. If the selected tool was not scored highest, the software 
engineer should have an opportunity to review the tool characteristics 
thoroughly to avold unexpected Installation difficulties. The selection 
concludes the separate sequence for Informal procurement. Continue with event 
3. / 
B, Acquisition Activities for Formal Procurement 
Bl. Acquisition Plan 


The plan generated here must Include all elements mentioned under Al plus the 
constralnts on the procurement process (e. g., set-aside for high labor surplus 
areas) and the detalled responsibilities for all procurement documents 
(statement of work, technical and administrative provisions In the Request for 
Proposal, etc.). 


B2. Technical Requirements Document 


The technical requirements document Is an Informal description of the tool 
requirements and the constraints under which the tool Has to operate. It will 
utilize much of “the matertal from the acquisition plan but should.add enough 
detall to support a meaningful review by the tool user. 


B3s User Review of Requirements 


The user reviews the technical requirements for the proposed procurement. As In 
“the case of event A4, the user may need to be prompted with pertinent questions, 
and there should be close management follow up In order to get a ‘timely 
response, - 


B4. RFP Generation 


aad \ 
From the technical requirements document and the user comments on It, the 
technical portions of the RFP can be generated, Usually these Include: 


‘ 


a . a - 

1. Aspecification of the tool as delivered. This should Include 
applicable documents, a definition of the operating environment, and 
the quality assurance provisions. 


36 


30 


2. A statement of work governing the tool procurement. This should 
state any applicable standards for the process by which the tool Is 
generated (e. g., configuration management of the tool), and 
documentation or test reports to be furnished with the tool. 
Training and operational support requirements are also Identified In 
the statement of work. 


Proposal evaluation criteria and format requirements. Evaluation 
criterla are listed In the approximate order of Importance. 
Subfactors for each may be Identified. Restrictions on proposal 
format (major headings, page count, desIred sample outputs) may also 
be Included. : 


B5. Solicitation of Proposals 

This activity Is carried out by administrative personnel. Capability IIsts of 
potential sources are malntalned by most purchasing organizations. Where the 
software organization knows of potential bidders, thelr names should be made 


known to the procurement office. When responses are recelved, they are screened 
for compilance with major legal provisions of the RFP. 


B6. Technical Evaluation 


Each of the proposals recelved In response to the RFP Is evaluated agalnst the 
criterla previously established. Fallure to meet major technical requirements - 
can lead to outright disqualification of a proposal.- Those deemed to be In "the 
competitive range" will be assigned point scores that will then be used together 
with cost and schedule factors that are belng separately evaluated by 
administrative personnel. 4 


B7. Source Selection r 


On the basis of the combined cost, schedule, and technical factors, a source for 
the tool Is selected. If this'was not the highest rated technical proposal, 
prudent management will require additional reviews by software management and 
the software engineer to determine that It Is Indeed acceptable. 


The source selection concludes the separate sequence for formal procurement. 
Continue with event 3. . : 


+ 


3. Procure Tool 


In addition to determining that the cost of the selected tool Is within the 
approved budget, the procurement process will also consider the adequacy of © 
Ilcensing and other contractual provisions and compliance with the “fine print" 
associated with all Government procurements. The vendor's responsibility for 
furnishtyg the source program, for meeting specific test and performance 
requirements, and for tool maintenance need to be Identified. tn Informal 
procurement, a perlod of trial use may be considered If this had not already 
taken pl under one of the previous events. 
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If the acquisition plan Indicates the need for outside training, the ability of 
the vendor to supply the training and the cost advantages from combined 
procurement of tool and tralntng should be Investigated. If substantial savings 
can be realized through simultaneous purchase of tool and training, procurement 
may be held up until! outside training requirements are defined (event 6). 


4. Evaluation Plan 


The evaluation plan Is based on the goals Identified In event 1 and the tool 
obectives derived from these In event 2. It describes how the attalnment of 
these objectives Is to be evaluated for the specific tool selected. Typical 
Items to be covered In the plan are milestones for Installation, dates and 
performance levels for the Initial operational capability and for subsequent 
enhancements, Where Improvements In throughput, response time, or turn-around 
time are expected, the reports from which these data are to be obtalned should 
be Identified, Responsibillty for tests, reports and other actions should be 
assigned In the plan. A topical outline of the Evaluation Report should be 


Thcluded. 


The procedure for the acceptance test Is a part of the Evaluation Plan, although 
In a major tool procurement It may be a separate document. It lists the 
detalled steps necessary to test the tool In accordance with the procurement 
provisions when It Is recelved, to evaluate the Interaction of the tool with the 
computer environment (e. g., adverse effects on throughput), and for generating 


an acceptance report. 


5. Toolsmithing Pfan 


The plan will describe the selection of the toolsmith, the responsibilities for 
the adaptation of the tool, and the training which will be required. The 

toolsmith should preferably be an experienced system programmer, famiiltar with 

the current operating system. Training In the operation and Installation of the 

selected tool In the form of review of documentation, visits to current users of 

the tool, or training by the vendor must be arranged, The toolsmithing plan Is 

listed here af an event for which the software engineer Is responsible, and In 

the dkscussfon of further events It Is assumed that the toolsmith will work 

under the direction of the software engineer. The toolsmithing plan should be 

approved by software management. L 


6. Iralning Plan 


The training pian should first consider the training Inherently provided with 
the tool, e. g., documentation, test cases, on-line diagnostics, HELP. These 
features may be supplemented by standard training alds supplied by the vendor 
for In-house tralning such as audio or video cassettes and lecturers. Because 
of the expense, training sessions at other locations should be consIldered only 
where none of the previous categories Is avallable. The number of personnel to 
recelve formal training should also be specified In the plan, and adequacy of 
In-house facI!ities (number of terminals, computer time, etc.) should be 
addressed, If training by the tool vendor Is desired, this should be Identified 
as early as possible to take permit training to be procured with the tool (see 
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step 3). User Involvement In the preparation of the training plan Is highly 
desirable, and coordination with the user Is considered essential. The training 
plan Is normally prepared by the software engineer and approved by software 
management, Portions of the plan should be furnished to procurement staff If 
outside personnel or facilities are to be utliized. 


7. ool Recelved 


The tool Is turned over by the procurlig organization to the software engIneer. 


8. Acceptance Test 


The software engineer or staff test the tool. This Is done as much as possible 
In an "as recelved" condition with only those modtfications made that are 
essential for bringing It up on the host computer. A report on the test*Is 
Issued. After approval by software management It constitutes the officlal 
acceptance of the tool. 


9. Orlentation 


When It has been ‘determined that the tool has been recelved In a satisfactory 
condition, software management holds an orlentation meeting for all personne! 
Involved In the use of the tool and tool: products (reports.or IIstings generated 
by the tool). The maln:purpose Is to communicate as directly as possIble the 
objectives of the tool use, such as Increased throughput or Improved legtbility 
of IIstings. Highlights of the ‘evaluation plan should ‘also be presented, and 
any changes In dutles assoclated with the Introduction of- the tool should be 
described. Personnel should be reassured that allowance will be,made for 
problems encountered during the Introduction, and that the full benefits of the 
tool may not make themse | ves’ felt for some time, 


10. Modifications ” 


This step Is carried out by the toolsmith ‘In accordance with, the approved 
toolsmithing plan. It Includes modi fications-of the tool Itself, of the 
documentation, and of the operating system. in rare cases some modification of 
the .computer proper may also be'necessary (channel assignments, etc.). Typical 
tool modifications: Involve deletion of unused options, changes In prompts’ or 
diagnostics, and other adaptatdons made for efficient use In the prevatling 
environment. ‘ Documentation of the modifications Is ar essential part of this 
everits : / 


“on 


- 
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Vendor literature for the tool Is reviewed In detall and Is talloréd for the 
prevalling computer env! ronment and for the tool modifications which have been 
made, Deleting: sections which are not-applicable can be Just as useful as 
adding material that Is required for the specific programming environment, 
Unused options shall be clearly marked or removed from the manuals, If there Is , 
some resident software for which. the tool should not be uséd (e. g., because of 
fanguage Incompatibility or confilcts In the operating system Interface), | 
warning notices alee ‘be Inserted Into the tool manual. ° 
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11. Training aS ae 


Training Is a jolnt responsibility of the software engineer and the tool user. 
The former Is responsible for the content (In ac¢ordance’ with the approved 
training plan), and the latter should have control over the length and 

‘ spheduling of sessions. Tralning Is an exceklent opportunity to motivate the 
user to utilize the tool. The tool user should-have the priyllege of 
terminating steps In the tralning that- are not helpful and of’ extending portions — - 
that are helpful but In which greater, depth Is desIred. Training Is not a 
one-time activity. Retraining or training In the use of additional’ options 
after the Introductory perlod Is.desIrable. This also provides an opportunlty 
for users to talk about problems with the tool. 


12, Use In the Operating Environment — 


The first use of the tool In the operational environment should Involve the most 
quallfled user personnel and minimal use of options. The first use should not 
be on a project with tight schedule constraints. Any difficulties«resul ting 
from this use must be resolved before expanded service 1s Initiated. If the 
_ first Use Is successful, then use by edupntonn! personnel and use of further 
opr lens may commence, eo ow: : 


User comments on tratning, first use’ of the tool, and use of actendad 
capabllIties are prepared and furnished to the software engineer. Desired 
Improvements In the user Interface, speédsor format of response, and In 
utilization of computer resources are appropriate topics. Formal: comments may 
be sollcited shortly after the Initial use, Byler 6" ‘months, and again after 1 
year. 


ad Evaluation Report 


The software engineer prepares the Evaluation: Report, using the outline 

generated In event 4, The user comments and observations of the toolsmith form 

Important Inputs to thls document. Most of all, It must discuss how the general 

goals and the tool objectives were met.’ The report may Include, of course, . 

observations on the Installation and use of. the tool, cooperation recelved- fron® 
the vendor In Installation or training, and any other "lessons learned", Tool 

and host computer modifications shall be described In the report. IT may 

contain a section of comments useful to future users of the tool. The report Is 

approved by software management and preferably also by funding management. | 


14, Determine If Goals Are Met 
. Funding management receives the Evaluation Report and determines whether the 
goals establ|lshed In event 1 have been met. This determination shall be In 
. . writing and It shall Include: ; 
‘ . h 
’ Attainment of technical objectives, — 
Adherence to budget and other resource constraints. 
Timeliness of the effort. 


Cooperation from other agencies. 
Recommendations for future tool acquisitions. 
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WORKSHOP ON PHASING oF SOFTWARE TOOLS 
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NATIONAL: BUREAU OF STANDARDS 
, Lecture Room B, Administration Bullding . 
“Monday, 18 May 1981 


AGENDA 


Welcome to NBS’ | a "* My Branstad, NBS 


_- Software - Automation Project an ~ R. C. Houghton, NBS 


ners Nae of ‘the REP ESNOP 


“Coffee Break ~ 


Survey, of Software Too! Usage e H. Hecht, SoHaR 
Tools Introduction Exper lence a e! 
“NASA Langley’ . S. Voigt 
Naval Alr-Deyelopment Center ° . H. Stuebing 
“NASA Goddard. _ F. McGarry 
Lunch 


Guidelines for Phasing Software Tools H. Hecht, SoHaR 
Into Development Environments 


Discusslon Groups 


Coffee Break 


Event Sequence for Tool Introduction He Hecht, SoHaR 
Discusston Groups . 


Wrap-Up 


WORKSHOP ON PHASING OF SOFTWARE TOOLS 


PARTICIPANTS ~ 
Leo Bel tracchl ; Larry Lombando 
Nuclear Regulatory Commission. Rome Alr Development Center 
. : a * 
Martha Branstad x 4! Frank McGarry 
_Nattonal Bureau of Standards NASA Goddard Spaceflight Center 
Arthur F. Chantker . Albert Moy 
Federal Aviation Administration Federal Bureau of Investigations 
Loprat ne Duvall. Albrecht Neumann 
11T Research Institute : National Bureau of Standards 
Shella Frankel Pat Powel | 
National Bureau of Standards National Bureau’ of Standards 
Tony Green Carol Proctor. 
Federal Trade.Commission ltl inols Institute of Technology 
Herbert Hecht ‘Wray Sexson 
SoHaR Incorporated Defense Mapping Agency 
Terry Heldelberg _ At Sorkow!tz . 3 
Lawrence Livermore Laboratory Department of Housing & Urban DevIi.pmt. 
Larry Hoover - Janet Stearns 
CRC -- ~~ Defense Mapping Agency 
Raymond Houghton Henry G. SteubIng 
National Rirsas of Standards ; Naval Alr Development Center 
Arnold Johnson Susan Volgt 
Federal Computer Testing Center NASA Langley Research Center 
Linda Lawrie | : 
US Army - CERL 
REVIEWERS . 
Plo De Feo : Marvin Zelkow! tz 
NASA Ames Research Center “University of Maryland 


Patricla (Santon!) Oberndorf 
Naval Ocean Systems Center . 


43° 


